Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor echo plugin #5565

Merged
merged 5 commits into from
Jul 17, 2024
Merged

Refactor echo plugin #5565

merged 5 commits into from
Jul 17, 2024

Conversation

pingsutw
Copy link
Member

Tracking issue

NA

Why are the changes needed?

Trying to optimize the echo plugin

  • using non-threadsafe map to store start times
  • If the sleep duration is zero it still starts a goroutine

What changes were proposed in this pull request?

  • Add sync.Mutex to the plugin, and lock it before accessing taskStartTimes map
  • Directly write the input literals to the output location if SleepDuration is 0

How was this patch tested?

sandbox

Setup process

import typing
from click.testing import CliRunner

from flytekit import task, workflow, ImageSpec, conditional
from flytekit.clis.sdk_in_container import pyflyte
from flytekit.core.task import Echo

new_flytekit = "git+https://github.com/flyteorg/flytekit.git@5d6e9a0d2882d7484db682e065dfa269d8a02964"

custom_image = ImageSpec(
    registry="ghcr.io/flyteorg",
    packages=[new_flytekit],
    apt_packages=["git"],
)


echo = Echo(name="echo", inputs={"a": float}, outputs={"o0": typing.Optional[float]})


@task(container_image=custom_image)
def calculate_circle_circumference(radius: float) -> typing.Optional[float]:
    return 2 * 3.14 * radius  # Task to calculate the circumference of a circle


@task(container_image=custom_image)
def calculate_circle_area(radius: float) -> int:
    return 3.14 * radius * radius  # Task to calculate the area of a circle


@workflow
def wf(radius: float) -> typing.Optional[float]:
    return (
        conditional("shape_properties_with_multiple_branches")
        .if_((radius >= 0.1) & (radius < 1.0), last_case=False)
        .then(calculate_circle_circumference(radius=radius))
        # .elif_((radius >= 0.5) & (radius < 0.9))
        # .then(calculate_circle_area(radius=radius))
        # .else_()
        # .fail("0.10 is an outlier.......")
        .else_()
        .then(echo(a=radius))
    )

Screenshots

Screenshot 2024-07-16 at 2 47 52 PM
Screenshot 2024-07-16 at 2 47 59 PM

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

NA

Docs link

NA

pingsutw added 5 commits July 13, 2024 14:28
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Copy link

codecov bot commented Jul 16, 2024

Codecov Report

Attention: Patch coverage is 0% with 37 lines in your changes missing coverage. Please review.

Project coverage is 60.97%. Comparing base (81afb76) to head (0ae89ac).
Report is 122 commits behind head on master.

Files with missing lines Patch % Lines
flyteplugins/go/tasks/plugins/testing/echo.go 0.00% 37 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5565      +/-   ##
==========================================
- Coverage   60.98%   60.97%   -0.02%     
==========================================
  Files         796      796              
  Lines       51647    51661      +14     
==========================================
  Hits        31498    31498              
- Misses      17249    17263      +14     
  Partials     2900     2900              
Flag Coverage Δ
unittests-datacatalog 69.31% <ø> (ø)
unittests-flyteadmin 58.73% <ø> (ø)
unittests-flytecopilot 17.79% <ø> (ø)
unittests-flytectl 67.44% <ø> (ø)
unittests-flyteidl 79.06% <ø> (ø)
unittests-flyteplugins 61.76% <0.00%> (-0.09%) ⬇️
unittests-flytepropeller 57.42% <ø> (ø)
unittests-flytestdlib 65.68% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pingsutw pingsutw merged commit 0b39839 into master Jul 17, 2024
48 of 50 checks passed
@pingsutw pingsutw deleted the update-echo-plugin branch July 17, 2024 22:32
vlibov pushed a commit to vlibov/flyte that referenced this pull request Aug 16, 2024
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Vladyslav Libov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants